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Abstract 

Our  recent  work  on  the  modeling  human-like  multi-task  behaviors  has  led  to  unexpected 
insights  into  a  possible  mechanism  contributing  to  individual  differences  in  human 
performance.  We  begin  by  outlining  our  modeled  conflict  resolution  strategy  for  the 
graceful  sharing  of  perceptual,  cognitive,  and  motor  resources  among  multiple  in-process 
tasks.  We  then  discuss  how  that  same  mechanism  might  drive  individual  differences  in 
concurrent  task  execution.  The  modeled  strategy,  in  this  case  operating  on  subtask 
structure,  may  be  one  contributor  to  how  individual  operators  differentially  decompose 
and  interleave  concurrent  tasks.  We  further  suggest  that  the  same  mechanisms  and 
structures  might  evolve  over  time  as  an  operator  learns  to  concurrently  engage  multiple 
related  tasks.  Theory  building  for  multi-task  behaviors  as  explored  through  model 
development  has  opened  a  new  line  of  inquiry  into  potential  mechanisms  for  individual 
differences  in  performance. 

Summary 

In  grounding  human  performance  modeling  in  a  theory  instantiated  in  an  architecture  we 
have  sought  to  not  only  produce  human-like  behaviors  in  a  model  but  also  provide  insight 
into  the  assembly  and  functioning  of  the  individual  capabilities  that  come  together  to 
produce  these  behaviors.  Much  of  the  research  effort  has  been  directed  at  understanding 
and  reproducing  expert  behaviors  at  workplaces  that  demand  concurrent  execution  of 
multiple  tasks.  What  we  have  found  surprising  is  that  the  mechanisms  that  we  have  in 
place  to  address  resource  contention  in  multitasking  have  provided  the  basis  for  new 
insights  into  the  mechanisms  that  we  now  suggest  may  be  important  contributors  to 
individual  differences  in  performance. 

We  begin  with  an  introduction  to  the  architecture  and  implementation  framework  for  our 
models.  The  discussion  is  restricted  to  the  essential  elements  of  the  architecture  that 
support  the  modeling  of  multi-task  performance.  We  then  provide  a  brief  overview  of  the 
unmanned  aerial  vehicle  (UAV)  environment  with  a  focus  on  the  role  of  the  sensor 
operator  (SO).  With  the  necessary  background  in  place,  the  mechanisms  within  the  model 
that  lead  to  individual  differences  in  performance  are  examined.  The  final  section  briefly 
outlines  possible  future  directions  for  the  research  effort. 

Modeling  Human  Performance  in  D-OMAR 

To  understand  how  it  was  possible  to  gain  insight  into  possible  mechanisms  leading  to 
individual  differences  in  performance  we  will  start  by  providing  an  overview  of  the  basic 
elements  of  the  models  we  have  been  building  and  the  theory  and  architecture  that 


underlies  the  models.  We  will  look  most  closely  at  how  the  architecture  supports  the 
human-like  multi-task  behaviors  of  the  models. 

The  models  have  been  used  to  represent  small  teams  such  as  commercial  aircrews,  air 
traffic  controllers,  or  UAV  operators.  Each  modeled  team  member  is  typically  seated  at  a 
complex  workplace  executing  procedures  that  require  coordination  with  other  team 
members.  From  a  bottom-up  perspective,  the  models  are  dependent  on  suite  of  perceptual 
and  motor  capabilities.  Perceptual  capabilities  focus  on  vision  and  audition  with  limited 
processing  of  haptic  input.  Motor  requirements  focus  on  hand  and  eye  operations. 

From  a  top-down  perspective,  we  are  concerned  with  supporting  task  completion  in  the 
models.  Model  behaviors  are  defined  by  goals  and  procedures  in  which  the  goals  define 
the  objectives  being  undertaken  by  the  models.  A  goal  includes  a  plan  with  sub-goals  that 
decompose  the  top-level  goal  as  necessary.  The  actions  taken  by  the  model  are  expressed 
as  procedures  invoked  by  the  sub-goals  within  a  plan.  The  procedures  typically  have 
numerous  branch  points  to  address  contingencies  that  may  be  encountered.  As  defined  by 
this  framework,  the  objective  of  a  task  are  expressed  as  a  goal  with  a  plan  made  up  of 
sub-goals — the  execution  of  the  task  is  conducted  by  the  sub-goals’  procedures.  Goals 
and  procedures  are  elements  of  the  Simulation  Core  (SCORE)  language  used  to  define 
model  behaviors. 

Our  concern  has  been  to  model  expert  performance  in  which  many  of  the  operator’s  skills 
have  been  highly  automatized.  Workplace  situations  require  the  concurrent  processing  of 
multiple  tasks  and  much  of  our  research  has  been  concerned  with  how  human  error 
intrudes  in  highly  skilled  behaviors. 

We  will  look  at  the  mechanisms  in  the  models  that  are  thought  to  support  multiple  task 
behaviors  in  humans.  We  will  suggest  that  the  same  mechanisms  may  be  at  work  in 
producing  individual  differences  in  performance. 

In  modeling  the  concurrent  demands  of  multiple  tasks  it  is  necessary  to  impose  human¬ 
like  bounds  on  model  performance.  Perceptual  and  motor  capabilities,  hands  and  eyes, 
are  resources  that  in  most  circumstances  can  only  serve  one  task  at  a  time.  The  concurrent 
demands  of  two  tasks  each  requiring  a  coordinated  hand-eye  operation  requires  a  conflict 
resolution  strategy.  We  base  the  access  to  the  functional  elements,  hands  and  eyes,  on  the 
priorities  of  the  contending  parent  tasks.  A  proactive  action  to  set  a  new  UAV  altitude 
will  preempt  a  background  instrument  scanning  tasks.  The  background  scanning  task  will 
simply  resume  once  the  altitude  has  been  set.  If  we  think  of  the  goals  and  procedures  of  a 
task  as  forming  a  tree,  the  contention  is  seen  to  be  resolved  at  the  leaf  nodes  of  the  tree. 

Exactly  the  same  conflict  resolution  protocol  also  addresses  high-level  protocol  issues. 

On  a  commercial  flight  deck,  the  aircrew  will  interrupt  their  conversation  to  listen  to  a 
communication  from  the  air  traffic  controller.  The  tasks  associated  with  attending  to  the 
air  traffic  controller  are  established  with  a  higher  priority,  and  hence,  will  interrupt  the 
lower  priority  intra-crew  conversation. 


In  each  case,  a  new  procedure  encounters  a  running  procedure  with  which  it  conflicts.  If 
the  new  procedure  has  higher  priority,  it  will  continue  execution  and  the  running 
procedure  will  be  suspended.  Otherwise,  the  new  procedure  will  be  suspended  and  the 
running  procedure  will  continue  to  completion.  Notably  absent  is  the  requirement  for  an 
explicit  executive  reasoning  over  procedures  to  produce  expert  multi-task  behaviors. 

We  will  suggest  that  the  same  conflict  resolution  strategy  that  addresses  resource 
contention  and  high-level  protocols  in  multi-task  behaviors  may  also  play  a  role  in 
individual  differences  in  performance. 

The  UAV  Domain 

Unmanned  aerial  vehicle  (UAV)  operations  provided  the  domain  for  the  modeling  research.  The 
UAV  team  for  a  mission  included  an  aerial  vehicle  operator  (AVO),  a  sensor  operator  (SO),  and  a 
multi-function  operator  (MFO)  that  may  be  supporting  one  or  more  UAV  teams.  Our  focus  was 
on  the  SO  as  he  or  she  conducted  surveillance  of  at  a  commercial  airport  where  armed  agents 
were  loading  and  fueling  a  commercial  aircraft  in  preparation  for  departure.  The  SO  conducted 
the  observations  using  a  TV  camera  and  an  infra-red  (IR)  sensor. 

The  tasking  for  the  mission  was  governed  by  a  text  document  containing  the  Essential  Elements 
of  Information  (EEI)  that  defined  mission  objectives  and  provided  information  for  EEI 
processing.  In  the  scenario  as  developed,  there  were  six  EEIs:  (1)  identify  the  target  aircraft 
among  the  aircraft  at  the  airport;  (2)  count  the  armed  agents;  (3)  monitor  the  fueling  of  the 
aircraft;  (4)  monitor  the  loading  or  unloading  of  the  aircraft;  (5)  check  the  target  aircraft’s  engines 
for  start-up;  and  finally  (6)  conduct  surveillance.  The  SO  interpreted  the  requirements  of  the  EEIs 
and  conducted  the  necessary  sensor  operations.  Each  of  the  EEIs  was  accomplished  using  the  TV 
sensor  except  the  checking  of  the  target  aircraft’s  engines  for  start-up  that  required  the  use  of  the 
IR  sensor.  As  the  SO  processed  the  EEIs,  the  SO  communicated  his  or  her  findings  to  the  MFO 
and  AVO. 

Multi-tasking  by  the  Sensor  Operator 

We  can  begin  to  think  about  the  SO’s  execution  of  the  EEIs  by  looking  at  the 
characteristics  of  the  individual  EEIs  that  impacted  their  execution.  The  individual  EEIs 
range  from  notably  simple  (e.g.,  count  the  armed  soldiers)  to  potential  quite  complex — 
the  surveillance  of  the  airport  could  play  out  in  any  number  of  ways.  Some  of  the  EEIs 
were  completed  immediately  (e.g.,  counting  of  the  armed  agents),  while  most  of  the  EEIs 
involved  the  monitoring  of  events  that  had  an  indeterminate  timeframe.  Lastly,  there  were 
dependencies  among  the  EEIs.  The  target  aircraft  had  to  be  identified  before  any  of  the 
other  EEIs  could  be  pursued;  events  detected  in  executing  one  EEI  could  impact  the 
pursuit  of  another  EEI. 

There  is  a  nice  alignment  between  what  we  will  speak  of  as  a  task  and  the  operations 
demanded  of  the  SO  as  he  or  she  worked  through  an  individual  EEI — each  a  well- 
defined,  notably  compact  unit  of  work,  allowing  that  some  are  not  immediately 
completed.  (The  exception  is  the  surveillance  EEI  that  has  been  decomposed  into  several 
concurrent  tasks  in  the  model.)  Hence,  in  terms  of  the  SO’s  work,  the  processing  of  an 
EEI  constitutes  a  task  guided  by  the  goal  of  completing  the  objective  defined  by  the  EEI. 


The  tasks  of  an  EEI  then  include  the  reading  of  the  textual  material  defining  each  task, 
mapping  the  task  defined  by  the  EEI  to  the  operations  to  be  performed,  and  the  execution 
of  the  required  operations.  Given  that  a  UAV  mission  will  typically  include  multiple 
EEIs,  we  can  now  broadly  define  bounding  approaches  to  accomplishing  the  necessary 
work.  A  first  approach,  what  we  will  term  the  read-process  approach,  can  be  defined  as 
read-process  in  the  sense  that  each  EEI  is  read  and  executed  in  turn.  At  the  other  extreme 
is  the  read-read  approach,  where  an  SO  might  read  all  the  EEIs  up  front  and  then  process 
them  with  much  concurrency. 

In  general,  the  read-process  approach  will  break  down  simply  because  the  SO  will 
encounter  EEIs  that  can  not  be  quickly  resolved  and  hence,  would  prevent  starting  the 
processing  of  subsequent  EEIs.  It  is  necessary  to  read  ahead  and  this  of  course  leads  to 
the  concurrent  execution  of  multiple  tasks.  Self  evident  in  its  shortcomings,  the  read- 
process  was  not  explored  using  the  model.  At  the  other  extreme,  the  read-read  approach 
was  explored  in  the  modeling,  followed  by  an  examination  of  the  trace  of  the  behaviors 
produced  that  showed  anomalies  in  task  execution.  The  exploration  of  the  aspects  of  the 
model  that  drove  the  middle  ground  in  behaviors  between  read-read  and  read-process  is 
discussed  in  the  next  section. 

Insights  into  Individual  Differences  in  Performance 

The  read-process  approach  to  EEI  processing  would  readily  have  been  established  using  conflicts 
between  the  reading  of  an  EEI  and  the  procedures  for  completely  processing  each  EEI.  An  EEI 
would  be  read  and  the  processing  initiated.  The  established  conflict  would  prevent  the  reading  of 
the  next  EEI  until  processing  of  the  first  EEI  was  completed.  It  would  have  been  a  dense  conflict 
structure  leading  to  highly  constrained  and  unsatisfactory  behaviors. 

What  this  initial  finding  suggests  is  that  the  structure  of  the  procedure  hierarchy  with  respect  to 
conflicts  between  procedures  drives  the  fine  structure  of  task  processing  in  important  ways. 
Changes  in  the  conflict  structure  for  procedures  leads  to  changing  patterns  of  task  execution.  A 
richer  structure  for  conflicts  yields  a  more  rigid  and  more  orderly  execution  of  a  task  by  inhibiting 
interruptions.  What  this  counter  intuitively  suggests  is  that  simpler  conflict  structures  enable  more 
complex  fine-grained  interleaving  of  competing  tasks — the  broad  range  of  variation  is  suggestive 
of  what  might  drive  individual  differences. 

The  populating  of  the  conflict  structure  previously  focused  on  regions  closer  to  the  roots  of  the 
goal-procedure  trees  to  establish  protocols  and  closer  to  the  leaves  to  govern  access  to  resources. 
What  is  interesting  is  that  our  attention  has  been  drawn  to  the,  until  now,  neglected  middle  ground 
of  the  procedure  trees.  We  modeled  a  sparse  conflict  structure,  however,  examination  of  the 
traces  of  the  trials  with  this  relaxed  structure  uncovered  some  questionable  model  behaviors.  A 
model  would  read  a  latitude  (for  pointing  the  sensor  package)  from  an  EEI,  dial  in  the  latitude  at 
the  console  and  then  jump  off  to  an  unconnected  step  in  the  processing  of  another  EEI,  maybe 
even  reading  a  new  EEI,  before  returning  to  set  up  the  longitude  associated  with  the  sensor 
pointing  operation.  The  behaviors  were  not  wrong;  they  were  just  not  the  likely  behaviors  of  a 
good  operator. 

As  we  pursued  the  process  of  examining  the  middle  levels  of  the  conflict  structure  and  adding, 
removing,  or  adjusting  the  pattern  of  conflicts,  we  uncovered  a  broad  range  through  which 


procedures  for  multiple  tasks  could  be  interleaved.  What  is  new  is  the  conflict  structure’s 
potential  role  in  establishing  individual  differences  in  task  execution.  The  findings  suggest  the 
particular  structure  of  the  conflicts  as  a  driver  of  these  individual  differences.  Moreover,  there  is 
the  suggestion  that  changes  in  the  conflict  structure  over  time  might  be  one  aspect  of  an 
individual’s  progress  in  learning  to  more  readily  and  robustly  achieve  the  successful  completion 
of  multiple  ongoing  tasks.  We  might  start  the  learning  of  a  new  set  of  tasks  with  a  complex,  dense 
conflict  structure  and  through  relaxation  of  the  structure — the  selective  removal  of  conflicts — 
learn  a  more  sophisticated  processing  of  multiple  ongoing  tasks. 

Next  Steps 

Lastly,  we  will  outline  future  directions  for  examining  model-based  insights  into 
individual  differences  in  performance. 
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